# Deploying Dagster on Helm

## Overview

[Kubernetes](https://kubernetes.io/) is a container orchestration system for automating deployment,
scaling, and management of containerized applications. Dagster uses Kubernetes in combination with
[Helm](https://helm.sh/), a package manager for Kubernetes applications. Using Helm, users
specify the configuration of required Kubernetes resources to deploy Dagster through a
[values file or command-line overrides](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing).
References to `values.yaml` in the following sections refer to
[Dagster's `values.yaml`](https://github.com/dagster-io/dagster/blob/master/helm/dagster/values.yaml).

Dagster publishes a [fully-featured Helm chart](https://github.com/dagster-io/dagster/tree/master/helm)
to manage installing and running a production-grade Kubernetes deployment of Dagster. For each Dagster component in the chart, Dagster
publishes a corresponding Docker image on [DockerHub](https://hub.docker.com/u/dagster).

## Prerequsites

`kubectl` should be configured with your desired Kubernetes cluster. You should understand
[the basics of Helm](https://helm.sh/docs/), and Helm 3 should be installed. If you are creating
your own user code images, Docker should be installed as well.

## Deployment Architecture

<!-- https://excalidraw.com/#json=4680957890134016,q6NWURUuPP_VThmbRQ89Jg -->

![dagster-kubernetes-default-architecture.png](/images/deploying/dagster-kubernetes-default-architecture.png)

### Components

<table>
  <tr style={{ background: "#F8F8F8" }}>
    <th>Component Name</th>
    <th>Type</th>
    <th>Image</th>
  </tr>
  <tr>
    <td>Daemon </td>
    <td>
      <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">
        Deployment
      </a>
    </td>
    <td>
      <a href="https://hub.docker.com/r/dagster/dagster-celery-k8s">
        dagster/dagster-celery-k8s
      </a>{" "}
      <i>(released weekly)</i>
    </td>
  </tr>
  <tr style={{ background: "#F8F8F8" }}>
    <td>Dagit</td>
    <td>
      <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">
        Deployment
      </a>{" "}
      behind a{" "}
      <a href="https://kubernetes.io/docs/concepts/services-networking/service/">
        Service
      </a>
    </td>
    <td>
      <a href="https://hub.docker.com/r/dagster/dagster-celery-k8s">
        dagster/dagster-celery-k8s
      </a>{" "}
      <i>(released weekly)</i>
    </td>
  </tr>
  <tr>
    <td>Database</td>
    <td>PostgreSQL</td>
    <td>
      {" "}
      <a href="https://hub.docker.com/_/postgres">postgres</a> <i>
        (Optional)
      </i>{" "}
    </td>
  </tr>
  <tr style={{ background: "#F8F8F8" }}>
    <td>Run Worker</td>
    <td>
      <a href="https://kubernetes.io/docs/concepts/workloads/controllers/job/">
        Job
      </a>
    </td>
    <td>
      User-provided or{" "}
      <a href="https://hub.docker.com/r/dagster/user-code-example">
        dagster/user-code-example
      </a>{" "}
      <i>(released weekly)</i>{" "}
    </td>
  </tr>
  <tr>
    <td>User Code Deployment</td>
    <td>
      <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">
        Deployment
      </a>{" "}
      behind a{" "}
      <a href="https://kubernetes.io/docs/concepts/services-networking/service/">
        Service
      </a>
    </td>
    <td>
      User-provided or{" "}
      <a href="https://hub.docker.com/r/dagster/user-code-example">
        dagster/user-code-example
      </a>{" "}
      <i>(released weekly)</i>{" "}
    </td>
  </tr>
</table>

### Daemon

The daemon periodically checks the Runs table in PostgreSQL for Pipeline Runs in that are ready to be launched.
The daemon also runs the [dagster-native scheduler](/deployment/dagster-daemon), which has exactly-once guarantees.

The Daemon launches the run via the <PyObject module="dagster_k8s" object="K8sRunLauncher" />, creating a Run Worker
[Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/) with the
image specified in the User Code Deployment.

### Dagit

The Dagit webserver communicates with the User Code Deployments via gRPC to fetch information needed to populate the Dagit UI.
Dagit does not load or execute user-written code to ensure robustness, and will remain available even when user code contains errors.
Dagit frequently checks whether the User Code Deployment has been updated; and if so, the new information is fetched.

Dagit can be horizontally scaled by setting the `dagit.replicaCount` field in the `values.yaml`.

By default, it is configured with a <PyObject module="dagster_k8s" object="K8sRunLauncher" />, which
creates a new Kubernetes Job per pipeline run.

### Database

The user can connect an external database (i.e. using a cloud provider's managed database service, like RDS) or run PostgreSQL on Kubernetes. This
database stores Pipeline Runs, Events, Schedules, etc and powers much of the real-time and historical data visible in Dagit.
In order to maintain a referenceable history of events, we recommend connecting an external database for most use cases.

### Run Worker

The Run Worker is responsible for executing the solids in topological order. The Run Worker uses the same image as the User Code Deployment at the
time the run was requested. The Run Worker uses ephemeral compute, and completes once the run is finished. Events that occur during the run are written to
the database, and are displayed in Dagit.

The Run Worker jobs and pods are not automatically deleted so that users are able to inspect results. It is up to the user to delete old jobs and pods
after noting their status.

### User Code Deployment

A User Code Deployment runs a gRPC server and responds to Dagit's requests for
information (such as: "List all of the pipelines in each repository" or "What is the dependency structure of pipeline X?").
The user-provided image for the User Code Deployment must contain a
[repository definition](/concepts/repositories-workspaces/repositories)
and all of the packages needed to execute within the repository.

Users can have multiple User Code Deployments. A common pattern is for each User Code Deployment to correspond to a different repository.

This component can be updated independently from other Dagster components, including Dagit. As a result,
updates to repositories can occur without causing downtime to any other repository or to Dagit.
After updating, if there is an error with any repository, an error is surfaced for that repository within Dagit; all other repositories and Dagit will still operate normally.

## Walkthrough

### Build Docker image for User Code

_Skip this step if using Dagster's example User Code image [dagster/user-code-example](https://hub.docker.com/r/dagster/user-code-example)._

Create a Docker image containing the Dagster repository and any dependencies needed to execute the objects in the repository. For reference, here is an example
[Dockerfile](https://github.com/dagster-io/dagster/blob/master/python_modules/automation/automation/docker/images/k8s-example/Dockerfile)
and the corresponding [User Code directory](https://github.com/dagster-io/dagster/tree/master/examples/deploy_k8s/example_project).

### Push Docker image to registry

_Skip this step if using Dagster's example User Code image._

Publish the image to a registry that is accessible from the Kubernetes cluster, such as AWS ECR or DockerHub.

### Add the Dagster Helm chart repository

The Dagster chart repository contains the versioned charts for all Dagster releases. Add the remote
url under the namespace `dagster` to install the Dagster charts.

    helm repo add dagster https://dagster-io.github.io/helm

### Configure your User Deployment

Update the `userDeployments.deployments` section of the Dagster chart's `values.yaml` to include your deployment.

The following snippet works for Dagster's example User Code image.

```yaml
userDeployments:
  enabled: true
  deployments:
    - name: "k8s-example-user-code-1"
    image:
      repository: "docker.io/dagster/user-code-example"
      tag: latest
      pullPolicy: Always
    dagsterApiGrpcArgs:
      - "-f"
      - "/example_project/example_repo/repo.py"
    port: 3030
```

The `dagsterApiGrpcArgs` field expects a list of arguments for `dagster api grpc` which is run upon Deployment creation and
starts the gRPC server. To find the applicable arguments, [read here](/concepts/repositories-workspaces/workspaces#running-your-own-grpc-server).

### Install the Dagster Helm chart

Install the Helm chart and create a release. Below, we've named our release `dagster-release`.
We use `helm upgrade --install` to create the release if it does not exist; otherwise, the
existing `dagster-release` will be modified:

    helm upgrade --install dagster-release dagster/dagster -f /path/to/values.yaml

Helm will launch several pods including PostgreSQL. You can check the status of the installation with `kubectl`.
If everything worked correctly, you should see output like the following:

    $ kubectl get pods
    NAME                                              READY   STATUS    RESTARTS   AGE
    dagster-dagit-645b7d59f8-6lwxh                    1/1     Running   0          11m
    dagster-k8s-example-user-code-1-88764b4f4-ds7tn   1/1     Running   0          9m24s
    dagster-postgresql-0                              1/1     Running   0          17m

### Run a pipeline in your deployment

After Helm has successfully installed all the required kubernetes resources, start port forwarding
to the Dagit pod via:

    export DAGIT_POD_NAME=$(kubectl get pods --namespace default \
      -l "app.kubernetes.io/name=dagster,app.kubernetes.io/instance=dagster,component=dagit" \
      -o jsonpath="{.items[0].metadata.name}")
    kubectl --namespace default port-forward $DAGIT_POD_NAME 8080:80

Visit http://127.0.0.1:8080, navigate to the [playground](http://127.0.0.1:8080/workspace/example_repo@k8s-example-user-code-1/pipelines/example_pipe/playground),
select the `default` preset, and click _Launch Execution_.

You can introspect the jobs that were launched with `kubectl`:

    $ kubectl get jobs
    NAME                                               COMPLETIONS   DURATION   AGE
    dagster-run-c8f4e3c2-0915-4317-a168-bf8c86810fb2   1/1           4s         6s

Within Dagit, you can watch pipeline progress live update and succeed!

## Conclusion

We deployed Dagster, configured with the default <PyObject module="dagster_k8s" object="K8sRunLauncher" />,
onto a Kubernetes cluster using Helm.
